A Deep Learning Approach to Safeguard Coral Reefs: Detecting Crown-of-Thorn Starfish in Underwater Footage

Part 1/2: EDA & Data Augmentation

By: Ken K. Hong, Oanh Doan

Abstract:

The Great Barrier Reef, a UNESCO World Heritage site, faces severe threats from climate change, environmental pollution, and the overpopulation of coral-eating starfish (COTS).

This project focuses on enhancing real-time COTS detection in underwater footage using a deep learning-based object detection model, YOLOv11. The model was trained on a dataset of 23,501 underwater images with bounding box annotations. It achieved:

With an inference speed of 7.0 ms per image (143 frames per second), the model shows satisfactory performance for real-time COTS detection, contributing to coral reef protection efforts.

Introduction/Background/Motivation

The Great Barrier Reef is the world’s largest coral reef ecosystem and a UNESCO World Heritage site, hosting over 400 coral species. It is one of Earth’s richest and most complex ecosystems. However, due to various factors—such as climate change, environmental pollution, seawater warming, and attacks by coral-eating starfish (COTS)—this coral reef ecosystem faces severe threats [5].

In particular, the overpopulation of COTS in Australia’s Great Barrier Reef has caused significant damage to the coral and poses a serious threat to its delicate ecosystem.

Objective

Our primary goal is to develop an object detection model using deep learning to assist in the real-time detection of COTS from underwater footage. Key steps in the project include:

A successful implementation of this model would contribute to the efforts to detect and control COTS outbreaks, protecting Australia’s marine life.

Previous Methods

Before deep learning-based models, techniques like the ‘Manta Tow’ [1] were used to detect and remove COTS. Developed in the 1960s, the Manta Tow method relies on:

While effective, these methods face several limitations:

Dataset Description

Our project uses the Crown-of-Thorns Starfish (COTS) dataset, compiled through a collaboration between CSIRO’s Data61, CSIRO Oceans & Atmosphere, Queensland University of Technology, and Google [2]. Key features of the dataset include:

Unique Characteristics of the Dataset

  1. Single Class: Focuses solely on detecting the COTS class.
  2. Sequential Annotations: Multiple images capture the same COTS from slightly different angles or positions.
  3. Cryptic Behavior: Includes overlapping COTS and partial visibility, as parts of the animal may be hidden.

Data Preprocessing and Transformation

In this section, we perform data preprocessing on the Great Barrier Reef dataset. This includes loading the training data, transforming and extracting relevant features such as image identifiers, annotations, and generating file paths. The transformations also involve counting annotations and preparing the dataset for further analysis and model training.

Exploratory Data Analysis (EDA) and Image Augmentation Visualization

In this section, we conduct an exploratory data analysis (EDA) to better understand the characteristics of the dataset, focusing on the impact of image augmentation techniques. By applying various image transformations, we visualize how these augmentations enhance the diversity and quality of training data, helping to improve model robustness and performance. This process aids in understanding the effects of data augmentation on model generalization, especially in the context of underwater imagery in the Great Barrier Reef dataset.

EDA

DATA AUGMENTATION

Import Modules

apply_rotation_transform

apply_random_affine_transform

apply_coarse_dropout

apply_grid_elastic_deformation

apply_pixel_dropout

apply_vertical_flip

apply_horizontal_flip

apply_hue_saturation_value_transform

apply_uniform_box_blur

apply_sun_flare

apply_zoom_blur

apply_random_brightness_contrast

show_all_data_augmentation

Result 1 | Video 1 - Frame 5487

Result 2 | Video 2 - Frame 5800

Reference:

[1] Australian Institute of Marine Science. Reef monitoring sampling methods, n.d. Accessed: 2024-12-07.

[2] Jiajun Liu, Brano Kusy, Ross Marchant, Brendan Do, Torsten Merz, Joey Crosswell, Andy Steven, Nic Heaney, Karl von Richter, Lachlan Tychsen-Smith, David Ahmedt-Aristizabal, Mohammad Ali Armin, Geoffrey Carlin, Russ Babcock, Peyman Moghadam, Daniel Smith, Tim Davis, Kemal El Moujahid, Martin Wicke, and Megha Malpani. The CSIRO Crown-of-Thorn Starfish Detection Dataset, 2021.

[3] Ultralytics Team. Hyperparameter tuning guide. https://docs.ultralytics.com/guides/hyperparameter-tuning/#what-are-hyperparameters, 2024. Accessed: 2024-12-09.

[4] Ultralytics. YOLOv11: Object detection and image segmentation models. https://docs.ultralytics.com/models/yolo11/, 2024.

[5] UNESCO World Heritage Centre. Great Barrier Reef, n.d. Accessed: 2024-12-07.

[6] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. CBAM: Convolutional Block Attention Module, 2018.